Journal of Clinical Epidemiology

Importance. Observational studies dominate evidence in cardiovascular outcomes research, yet no widely adopted reporting standard requires authors to disclose how much of an observed association is attributable to systematic error. The negative-control calibration framework of the Observational Health Data Sciences and Informatics consortium returns a binary answer: a calibrated P-value either crosses 0.05 or it does not. That answer conflates "is there a signal" with "is the signal credible." Objective. To define the Bias Attribution Fraction (BAF), a continuous estimator in [0, 1] that quantifies the fraction of an observed log relative risk that is attributable to systematic bias, together with its 95% credible interval, and to evaluate whether reporting calibrated P-value plus BAF plus interval width produces more defensible decisions than reporting the calibrated P-value alone. Design. Three-part study: (i) metric definition and interval construction; (ii) Monte Carlo simulation across 1920 conditions by 1000 repetitions = 1,920,000 simulated target estimates (a primary grid plus a sign-flipped companion grid of identical size, pooled for all analyses), with truth known by construction; (iii) two-arm beta-blocker target trial emulation in MIMIC IV, adhering to the TARGET guideline, with targeted maximum likelihood estimation (TMLE) and 12 negative-control outcomes. Setting. MIMIC IV (critical care) and MIMIC IV ECG, 2008 to 2019. Participants. Adults with a first intensive care unit stay and a recorded 12-lead ECG. Exposures. Two clinical questions are evaluated: (1) reaching at least 50% of guideline-recommended target dose of an index beta-blocker within 7 days of admission, sustained to 12 months, versus lower or no exposure; (2) receiving guideline-directed medical therapy (GDMT) with at least two of three foundational drug classes at adequate dose for 12 months versus fewer. Main Outcomes and Measures. One-year all-cause mortality. Primary analytic outputs: (i) the calibrated P-value from a negative-control empirical null distribution with 12 outcomes; (ii) BAF-hat with a Markov chain Monte Carlo 95% credible interval; (iii) a verdict drawn from a truth-calibrated lookup table mapping the BAF-hat by interval-width cell to a probability of being bias-dominated. Results. Across 1,920,000 simulated repetitions, the calibrated P-value alone yielded a misuse rate (declared usable while truth was bias-dominated) of 36.02%. Adding the BAF point estimate reduced misuse to 16.48%, and requiring the whole credible interval of BAF to lie below 0.5, clear of the bias-dominated zone, reduced misuse to 10.16%. Requiring in addition a calibrated probability of bias dominance below 15% reduced misuse to 3.02% at the cost of declaring 22.1% of estimates usable (versus 61.8% for the calibrated P-value alone). Read from the bias-dominated view, meaning that withholding a repetition is a positive call and being truly bias-dominated is a positive truth, the BAF estimator achieved an area under the receiver operating characteristic curve of 0.922, compared with 0.787 for the calibrated P-value. In MIMIC IV, question 1 produced a calibrated P-value of 0.894 and BAF-hat = 0.875 (95% credible interval 0.385 to 0.992); the apparent one-year mortality reduction of 16.9% was entirely attributable to systematic bias and the association was not usable as effect evidence. Question 2 also failed layer 1, with a calibrated P-value of 0.1026, and BAF-hat = 0.460 placed it in the "competitive, no verdict" zone. Conclusions and Relevance. Pairing the calibrated P-value with BAF and its 95% credible interval turns a binary credibility judgement into a three-channel report (signal existence, effect size, and bias fraction). Despite TARGET-adherent design, doubly robust TMLE, and 12 negative controls, both clinical questions in this study still failed to qualify as effect evidence, with question 1 demonstrating complete bias domination and BAF quantifying the systematic error that remained. The reporting standard proposed here can be applied to any observational study that includes negative controls, without requiring additional data.

BackgroundCochrane Risk of Bias 2 (RoB 2) assessment is methodologically demanding and resource intensive. Large language models may assist this process but can produce variable judgments. AimTo develop a rule-constrained LLM system for RoB 2 assessment and evaluate its temporal stability and agreement with human reviewers. MethodsRoB2-Evaluator was developed using GPT-5.2 with fixed RoB 2 guidance and a locked instruction framework. Seven RCTs were used for calibration before version freezing. The frozen system was evaluated on 10 separate RCTs, with repeat assessment after one week and comparison against consensus judgments from two independent human reviewers. ResultsTemporal agreement was 90.0% (weighted {kappa} = 0.93), and agreement with human consensus was also 90.0% (weighted {kappa} = 0.93). Methodological rationales were stable in 94.3% of repeated comparisons. Full AI-human conceptual alignment occurred in 85.7%. ConclusionRoB2-Evaluator showed encouraging post-freeze stability, human agreement, and rationale consistency, supporting larger independent evaluations.

Objectives. Observational studies report large, conflicting associations between common drugs and hip or knee replacement. We tested them in the serious adverse event (SAE) tables of randomized trials on ClinicalTrials.gov. Methods. We included randomized drug trials with at least 500 participants. A language model classified arms from summaries without outcome counts (masked by instruction only). We pooled osteoarthritis-related SAEs by drug class as Mantel-Haenszel risk ratios (RRs). Because the plan followed a preliminary analysis of 72 trials, we report estimates rather than tests, checked by positive and negative controls and calibration. Results. The 881 placebo- or usual-care-controlled trials included 1.75 million participants and 9,408 osteoarthritis-related SAEs. The tables reproduced CANTOS's own canakinumab effect (RR 0.57, 95% CI 0.39 to 0.82), the anti-nerve growth factor excess (2.31, 1.82 to 2.92) and two of three modest effects specified before extraction: gallbladder disease with glucagon-like peptide-1 receptor agonists (GLP-1RAs; 1.36, 1.22 to 1.51) and ketoacidosis with sodium-glucose cotransporter-2 (SGLT2) inhibitors (1.96, 1.48 to 2.60). In the same trials, osteoarthritis-related SAEs did not differ detectably with GLP-1RAs (0.97, 0.86 to 1.10; calibrated 0.87, 0.54 to 1.41) or SGLT2 inhibitors (0.96, 0.82 to 1.12; calibrated 0.86, 0.53 to 1.40). Other proposed drugs gave imprecise estimates. Post hoc, 8 of 21 published observational estimates differed from randomized ones, in both directions (3 to 6 after calibration). Conclusions. Over about two years, GLP-1RAs and SGLT2 inhibitors had no detectable effect on osteoarthritis-related SAEs in trials that detected their known effects. Slow or modest structural effects remain possible.

Purpose: We introduce the Temporal Index of Divergent Evidence (TIDE), a metric that evaluates how the topic center of gravity of a corresponding corpus of literature aligns with, or shifts relative to, a recommendation's semantics over time. Methods: TIDE computes annual mean cosine similarity using MedCPT embedding vectors between a recommendation and corresponding PubMed records binned by year (2000-2024), testing for monotonic temporal trend (Kendall {tau}). Three recommendations with expected divergent, stable, and convergent trajectories were examined: two-stage exchange for periprosthetic joint infection (PJI), emergent fasciotomy for acute compartment syndrome (ACS), and non-operative management of distal radius fractures (DRF) in adults [≥] 65 years. The signal was characterized by an additive per-token attribution, an isolated vs. in-context analysis, and an adversarial minimal-pair test. Results: PJI diverged ({tau} = -0.84, year-shuffle p = 0.001), DRF converged ({tau} = +0.42, p = 0.005), and ACS showed no detectable trend ({tau} = -0.07, p = 0.66). The two significant directions (PJI, DRF) held under matched input [CLS] pooling, while ACS remained near zero. Publication volume baselines did not distinguish the three trajectories (publication count was universally positive, {tau} = +0.77 to +0.97). A reversed-meaning minimal-pair control showed little cosine sensitivity to reversal. Paper-level TF-IDF recovered the qualitative three-case pattern. Conclusion: Embedding-based drift analysis is a proof-of-concept candidate pre-triage signal that tracks topical alignment, not evidentiary direction; superiority over TF-IDF remains unproven. The present study is hypothesis-generating, showing potential use of embedding representations as an analytical unit rather than simply for retrieval.

Background: Large language models (LLMs) can support systematic reviews, but accurate individual outputs do not establish whether the final synthesis preserves the clinical question, accounts for statistical dependence and incorporates corrections. Objective: To develop and evaluate a framework linking LLM-assisted evidence processing to a versioned, auditable release of synthesis outputs. Methods: We used LLM agents to interpret sources and extract data. Deterministic code enforced statistical rules; investigators resolved material ambiguities and authorized release. We specified ten release properties covering evidence identities, statistical contributions and propagation of corrections. We retrospectively evaluated six integrity domains and historical failure events in one registered prognostic review, without an external comparator or held-out domain. Results: Fifty distinct root-cause events were documented, including 12 that had changed a pooled result before correction. Forty-six were resolved, and four remained disclosed limitations. The corpus comprised 454 reports, 445 studies, 441 cohort entities and 421 dependence clusters. Forty-one of 49 registered analyses were fitted, and eight retained explicit non-fitted states. All 39 source records across five principal analysis families reached a terminal source state. Two implementations within the project agreed across 1,217 numerical comparisons. All 94 file comparisons between release and publication packages were byte-identical. Two reviewers confirmed 39 principal records after seeing the same recommendations. Conclusions: This case provides a framework for inspecting synthesized evidence together with its provenance, statistical meaning and correction history. Comparative validity, generalizability and benefit in patient-centred care require independent evaluation.

BACKGROUND: Federal policy now requires sponsors to submit diversity action plans for clinical trials, yet the modifiable features of trial design associated with enrolled composition remain poorly characterized. English-language proficiency requirements are written into eligibility criteria and recorded as free text in a public registry, making them visible and, in principle, amenable to revision, but their relationship to enrolled composition has been examined only within single disease areas and in small samples of trials. We examined whether these requirements and documented language accommodation were associated with the racial and ethnic composition of enrolled participants. METHODS AND FINDINGS: We conducted a cross-sectional study of interventional trials registered on ClinicalTrials.gov with at least one US site and posted results, using a registry snapshot retrieved in October 2026. The primary cohort comprised the 44,487 trials with exclusively US sites, of which 21,924 (49.3%) reported baseline race and 17,434 (39.2%) reported baseline ethnicity with counts that reconciled with the posted denominator. Both exposures were classified from free-text eligibility criteria using regular expressions validated against independent physician review of 150 trials. For the English-language requirement classifier, sensitivity was 88.2% and specificity was 99.2%. The outcome was the trial-level proportion of enrolled participants in each category, analyzed using generalized linear models adjusted for intervention class, therapeutic area, sponsor, phase, enrollment, sites, start year, US Census region, and the population composition of the trial's site states. An English-language proficiency requirement appeared in 5,736 trials (12.9%); its prevalence rose from 7.4% of trials starting in 2010 to 26.2% in 2023 (odds ratio [OR] per decade 2.91, P<0.001), whereas documented accommodation, an eligibility statement allowing participation in a language other than English (Spanish in 99.9% of flagged trials), remained rare (788 trials, 1.8%). The requirement was associated with lower enrolled proportions of Asian (OR=0.85, 95% CI 0.77-0.93, P=0.001) and Hispanic/Latinx participants (OR=0.85, 95% CI 0.79-0.91, P<0.001), with no evidence of an association for White participants (OR=0.99, 95% CI 0.95-1.04, P=0.75). Documented accommodation was associated with a higher proportion of Hispanic/Latinx participants (OR=3.23, 95% CI 2.82-3.71, P<0.001). Geographic adjustment did not change the statistical significance of the requirement associations and attenuated the accommodation association. The main limitations were the cross-sectional design, which precludes causal inference; the restriction of enrollment analyses to trials reporting baseline demographics, which were more recent than those without such reporting; and possible misclassification of automatically classified exposures. CONCLUSIONS: English-language proficiency requirements became more than three times as common between 2010 and 2023 and were associated with lower enrolled proportions of Asian and Hispanic/Latinx participants. Less common documented language accommodation was associated with a higher enrolled proportion of Hispanic/Latinx participants. Because these criteria are investigator-written and publicly recorded, they may be a tractable target for improving enrollment equity, although whether revising them changes who enrolls requires prospective evaluation

Introduction: The TREOCAPA (Prophylactic TREatment Of the duCtus Arteriosus in Preterm infants by Acetaminophen) trial was a pan-European, multi-centre, phase II/III trial investigating the impact of prophylactic paracetamol (also known as acetaminophen) given during the first 5 days of life for children born before 29 weeks' gestational age (GA) on survival to 36 weeks' post-menstrual age without severe neonatal morbidities. However, due to time restrictions, long-term follow-up was not planned as part of the initial trial despite recommendations for neonatal trials to include endpoints measuring efficacy after discharge from the neonatal unit and on-going debate about the long-term effects of paracetamol in the neonatal period. This protocol describes a study to follow-up participants in the TREOCAPA trial at 2 years corrected age initiated after the trial's onset. Methods and analysis: The TREOCAPA Long Term (TREOCAPA-LT) study population consists of all children included in the TREOCAPA phase III trial in centres in countries participating in the follow-up study who are aged between 23.5 and 27.5 months corrected age during the study period. Families are contacted by the clinical teams and a parent or guardian is invited to complete an internet-based (online) questionnaire. The primary outcome is cognition, as measured by the non-verbal Cognitive Scale standard score from the Parent Report of Children's Abilities-Revised (PARCA-R) questionnaire. Secondary outcomes are moderate to severe neurodevelopment, health service use and costs. Data will be analysed according to the study groups to which children were randomised in the original TREOCAPA RCT. Ethics and Dissemination: The TREOCAPA-LT protocol was first approved by ethics' committees in France (the study's sponsor) and subsequently by all participating countries. Results from TREOCAPA-LT will be published in open access journals in accordance with the funder's regulations. Registration: https://clinicaltrials.gov/study/NCT06064825

Aims. It is unknown whether large language models (LLMs) can predict a trial's result at the design stage, from the information available when it is registered. We tested three LLMs prospectively on the Late-Breaking Science programme of the European Society of Cardiology (ESC) Congress 2026. Methods and results. Of 139 trials, 50 were selected by pre-specified criteria, and predictions were locked and registered on the Open Science Framework before the congress. Claude Fable 5, GPT-5.5 and Gemini 3.5 Flash, without web access, received each title and an investigator-compiled summary of the registered design. They gave the probability that the primary endpoint would be met, and a point estimate with an 80% prediction interval for a locked effect measure. Results were adjudicated from publications (19) or presenters' slides (31). Of 48 scorable trials, 28 (58%) were positive. The area under the receiver operating characteristic curve was 0.80 (95% CI 0.66-0.92) for Claude, 0.79 (0.66-0.91) for GPT and 0.74 (0.59-0.87) for Gemini, with accuracies of 73%, 71% and 67%. Among 26 trials whose effect was reported exactly as locked, the direction was correct in 88% of predictions and the median multiplicative error of ratio estimates was about 15%. The 80% prediction intervals contained the observed effect in 73.1% (Claude), 84.6% (GPT) and 57.7% (Gemini). Conclusion. From the registered design alone, LLMs showed statistically significant discrimination between positive and negative trials and usually predicted the direction and approximate size of the effect. Such predictions might help trial design, and further studies are needed.

Background Under-five mortality remains a major public health challenge in Zambia despite substantial reductions over recent decades. Demographic and Health Survey (DHS) data are commonly analysed using Cox proportional hazards models; however, violations of the proportional hazards assumption and unobserved cluster-level heterogeneity may compromise inference. This study compared advanced survival modelling approaches to determine the most appropriate framework for analysing under-five mortality using pooled Zambia DHS data. Methods We conducted a pooled secondary analysis of the 2018 and 2024 Zambia Demographic and Health Survey child recode datasets. The analysis included 9,982 children, among whom 266 died before their fifth birthday. A survey-weighted Cox proportional hazards model was fitted as the reference model and assessed using Schoenfeld residual diagnostics. To address identified methodological limitations, an extended Cox model, a shared-frailty model, and a Bayesian survival model were subsequently fitted. Model performance, assumptions, and the consistency of adjusted hazard ratio estimates were compared across modelling approaches. Results The global Schoenfeld test indicated violation of the proportional hazards assumption ({chi}2=51.26, p=0.002). In the adjusted extended Cox model, male children had higher mortality risk than females (aHR 1.95, 95% CI 1.40-2.73), while children with gestation [&ge;]9 months had lower mortality risk than those with shorter gestation (aHR 0.21, 95% CI 0.14-0.32). Average and large birth size were associated with lower mortality compared with small birth size (aHR 0.47, 95% CI 0.34-0.66 and aHR 0.57, 95% CI 0.39-0.84, respectively). Newborn health checks before discharge were associated with lower mortality (aHR 0.50, 95% CI 0.38-0.66), as was any antenatal care (aHR 0.25, 95% CI 0.09-0.75). A high-risk fertility pattern involving fourth or higher births with an interval [&ge;]24 months was also associated with lower mortality (aHR 0.57, 95% CI 0.41-0.80). The shared-frailty model produced similar estimates, including increased mortality among male children (aHR 1.39, 95% CI 1.08-1.78) and twins or multiple births (aHR 2.74, 95% CI 1.36-5.53). Bayesian estimates were broadly consistent with the extended Cox model. Conclusions Advanced survival models provided a more robust analytical framework than the conventional Cox model for evaluating under-five mortality in nationally representative DHS data. While the extended Cox model accommodated time-varying effects, the shared-frailty model captured unobserved cluster-level heterogeneity, and the Bayesian model confirmed the stability of parameter estimates. Incorporating these complementary approaches can improve the validity of survival analyses and strengthen evidence for child survival research and policy in low- and middle-income settings.

Introduction: Primary care electronic health records (EHRs) are an important resource for health research, but death recording in primary care EHRs may be incomplete or differ from Office for National Statistics (ONS) death registrations. We used OpenSAFELY-TPP to assess the completeness and concordance of death recording and provide recommendations for using date-of-death information. Methods: Using linked primary care EHRs and ONS death registrations, we assessed whether ONS-registered deaths from 2020-2025 had a primary care date of death and, where present, calculated the difference between dates. Where a primary care date was absent, we assessed death-related SNOMED CT codes. Analyses were conducted overall and by year, population subgroup, and general practice. We also examined GP registration end dates and primary-care-recorded deaths without a corresponding ONS record, including monthly patterns near the data extraction date. Results: Among 1,409,275 ONS-registered deaths, 88% had the same date recorded in primary care and 8% had no corresponding primary care date. Among 1,324,995 deaths recorded in primary care, <2% had no corresponding ONS record. Deaths recorded in only one source were more frequent among people aged <45 years and those from Mixed, Asian or Asian British, Black or Black British, and Chinese or Other ethnic groups. ONS-only dates were also more frequent in London and the West Midlands. Primary-care-only deaths increased close to date of data extraction, reaching 7% in February 2026. Same-day GP deregistration increased from 9% in 2020 to 85% in 2025. Conclusions: Death recording in primary care EHRs showed high completeness and concordance with ONS registrations, but discrepancies remained for recent deaths and some population groups. Data recency and population-specific completeness should therefore be considered when using linked mortality data for research.

Objective To determine the comparative effectiveness of different digital health interventions for adults with low back pain. Design Systematic review and network meta-analysis of randomised controlled trials. Data sources PubMed, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), CINAHL, APA PsycINFO, ClinicalTrials.gov, and the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP), from inception to 19 March 2026 (updated to 8 August 2026). Study selection Randomised controlled trials enrolling adults with low back pain. Interventions were classified as telemedicine, web-based applications (Web-App), mobile applications (Mobile-App), virtual reality (VR), exergaming, biofeedback devices, face-to-face care, and control. Paired reviewers independently screened records, extracted data, and assessed risk of bias using the Cochrane Risk of Bias 2 tool. Methods Frequentist random-effects network meta-analyses synthesised pain intensity and physical function (primary outcomes), and pain-related fear avoidance, health-related quality of life, self-efficacy, anxiety, and depression (secondary outcomes), assessed at post-intervention (from the end of treatment to < 2 months post-intervention), short-term ([&ge;]2 to <6 months), mid-term ([&ge;]6 to <12 months), and long-term ([&ge;]12 months) follow-up. Effects were reported as standardised mean differences with 95% confidence intervals. Transitivity was examined across potential effect modifiers, and Confidence in Network Meta-Analysis (CINeMA) informed confidence ratings. Results Ninety-two reports representing 84 trials involving 11 649 participants were included. At post-intervention, compared with control, VR showed the largest estimated reduction in pain intensity (SMD -1.43, 95% CI -1.87 to -0.98; moderate confidence) and improvement in physical function (-0.86, -1.45 to -0.26; very low confidence). VR was also associated with lower pain-related fear avoidance (-1.14, -1.68 to -0.61) and better health-related quality of life (0.97, 0.57 to 1.36) (both moderate confidence). Exergaming, Mobile-App, Web-App and telemedicine were also associated with lower pain intensity, while Mobile-App, Web-App, telemedicine, and biofeedback devices were associated with better physical function. For self-efficacy, exergaming (0.77, 0.05 to 1.49) showed favourable post-intervention effects, although confidence in this estimate was very low. No intervention showed a clear effect on anxiety, with confidence ranging from very low to moderate across comparisons, whereas telemedicine was associated with lower depression (-0.22, -0.44 to 0.00; low confidence). Networks became progressively sparse at later follow-up, and evidence of sustained effects was inconsistent across interventions and outcomes. Global inconsistency was detected in the post-intervention pain-intensity network, with local inconsistency identified for selected exergaming comparisons, and substantial heterogeneity was present in several networks. Conclusions Among adults with low back pain, VR showed the largest estimated post-intervention effects versus control for pain intensity, physical function, pain-related fear avoidance, and health-related quality of life, although comparative effects differed across outcomes. Self-efficacy showed favourable estimates for selected interventions, whereas evidence for anxiety and depression was limited. Evidence beyond treatment completion was increasingly sparse, and confidence in many network estimates was low or very low; firm comparative recommendations therefore remain premature. Systematic review registration PROSPERO CRD420261339841

Background and Objective: Adding belzutifan to adjuvant pembrolizumab improved disease-free survival (DFS) in LITESPARK-022, but its durability beyond the 1-year treatment is unknown. Methods: Following a registered plan, we reconstructed individual patient data from published Kaplan-Meier curves of ten phase 3 adjuvant renal cell carcinoma trials. Within each trial, we estimated restricted mean survival time differences ({Delta}RMST), hazard ratios (HRs) before and after planned treatment ended, and DFS differences. Key Findings and Limitations: In LITESPARK-022 (central review), the 24-month {Delta}RMST was 1.34 months (95% confidence interval 0.73-1.95), and the belzutifan arm remained ahead after month 12 ({Delta}RMST increment 0.74 months/yr, 0.32-1.19). The HR was 0.55 (0.43-0.71) before month 12 and 1.16 (0.87-1.54) afterwards (post/pre ratio 2.10); investigator assessment showed similar attenuation. Similar ratios occurred in KEYNOTE-564 at its first interim analysis (2.02), whose benefit persisted, and in SORCE and PROTECT (2.09, 2.59), whose benefit faded. From months 12 to 30 (post hoc), the DFS difference changed by -3.7 percentage points (-8.1 to 0.8) with central review and 0.6 (-4.0 to 5.0) with investigator assessment. Data were reconstructed, and follow-up was short. Conclusions and Clinical Implications: Relative hazard reductions with belzutifan were greater before month 12; durability of the absolute DFS advantage is uncertain and assessment-dependent, so single-HR projections should be avoided.

Background: As musculoskeletal physiotherapy evolves toward advanced practice and direct access, evaluating high levels of clinical competence has become pressing. Quality indicators underpin large-scale audit systems, but indicators discriminating advanced competence remain insufficiently defined. Objective: To identify and characterize indicators capable of discriminating high levels of clinical competence in musculoskeletal physiotherapy. Methods: A modified Delphi comprised an exploratory pre-round and two consensus rounds. Experienced French-speaking musculoskeletal physiotherapists were recruited using an expertise scoring grid. Candidate indicators, generated from open-ended responses and existing quality indicators, were rated on 6-point Likert scales for relevance, formulation, and tool adequacy. Consensus was predefined as [&ge;]75% of ratings [&ge;]5. Results: Thirty-two experts completed the pre-round, 29 the first round, and 27 the second (84% retention), generating 49 candidate indicators. Thirteen reached consensus for relevance, and 10 also for formulation. Retained indicators primarily reflected advanced clinical reasoning: real-time adaptation, hypothesis prioritization, decision-making under uncertainty, and error management. Only one outcome indicator reached consensus: clinical performance weighted by case complexity. Conclusion: High levels of clinical competence in musculoskeletal physiotherapy appear characterized less by standardized outcome attainment than by managing complexity, uncertainty, and error through advanced clinical reasoning. These findings may inform competence-oriented audit and feedback systems. Trials registration: Not applicable.

Background Large language models (LLMs) show promise in scientific literature reviews, but cloud-based proprietary models like GPT-4 raise concerns around privacy, copyright, and licensed article access. Using a scoping review on AI applications for treatment effect estimation as a use case, this study evaluated local open-source LLMs for automated article screening in scientific literature reviews and developed a practical workflow for their implementation. Methods We extracted a sample dataset of 300 records from an ongoing scoping review on machine learning methods for treatment effect estimation using real-world medical data. We divided the dataset into training (n = 200) and testing (n = 100) datasets. We evaluated 11 locally deployed open-source LLMs from five model families (GPT-OSS, Gemma, Qwen, Mistral, Llama) for both abstract screening and full-text screening. Full-text screening was done using text converted by two PDF-to-text conversion tools: Docling and PyMuPDF4LLM. Performance was assessed using raw accuracy with 95% Wilson's confidence interval, relaxed accuracy, Cohen's kappa, and runtime. During workflow development, prompt engineering, output extraction strategies, and generation settings were iteratively optimized using the training dataset. Results GPT-OSS-20B achieved the highest raw accuracy for title and abstract screening (0.890 on the training set and 0.850 on the test set), followed closely by Gemma-4-31B-it (0.875 on the training set and 0.830 on the test set) and Gemma-4-26B-A4B-it (0.810 on the training set and 0.790 on the test set). In full-text screening, slightly higher raw accuracy and reduced false-positive rates were observed, with GPT-OSS-20B and Gemma-4 variants consistently demonstrating strong performance. Only minor variations were found between the two PDF-to-text conversion methods evaluated, though runtime varied substantially across models. Based on these findings, we compiled a set of practical recommendations for local LLM-assisted article screening. Conclusions Local LLMs can effectively support literature screening when integrated into a carefully designed workflow. Recent generations of open-source local models achieved performance comparable to human reviewers and proprietary LLMs. Local deployment also provides advantages in data privacy, copyright compliance, customization, and cost-efficiency. While human oversight remains essential, the proposed workflow demonstrated that local LLMs represent a practical and reproducible approach for accelerating evidence synthesis.

Background. Large language models (LLMs) are increasingly used to automate title-abstract screening in systematic reviews, but validation studies typically report sensitivity, specificity, or Cohen's kappa: metrics that are unstable under the low and unbalanced prevalence that characterises most reviews. No previous study has simultaneously combined a medical topic; a hybrid two-stage pipeline (rules + LLM); a three-way classification with a separately sampled uncertain class; stratified sampling with three independent human reviewers; and dual reporting of prevalence-robust coefficients (PABAK and Gwet's AC1). Objective. To validate a two-stage automated screening pipeline (a rule-based Python script followed by a reproducible LLM adjudication of uncertain records) against three independent human reviewers in a medical systematic review on high on-treatment platelet reactivity (HTPR), aspirin resistance, and carotid plaque. The primary validation target is the Stage 1 automatic-exclusion decision (670/904 records), which is the workload the pipeline is designed to take off the human reviewer. Methods. From 904 screened records (Stage 1: 131 INCLUDE, 670 EXCLUDE, 94 UNCERTAIN; Stage 2 LLM via API, temperature = 0: 11 INCLUDE, 83 EXCLUDE), a stratified sample of 185 records (50 per Stage 1 stratum + 35 Stage 2; seed = 42) was judged independently and blind to the automated labels by three physicians: E.P. (pipeline designer), D.C. and P.P.B. (both external to the thesis project). Concordance was quantified with sensitivity, specificity, PABAK (q = 3 throughout), Gwet's AC1, work saved over sampling (WSS), and number needed to read (NNR), AI versus each reviewer and between reviewers, overall and by stratum. Majority labels (2 of 3) are reported as a supplementary panel; they do not replace pairwise tables as the primary reference for sensitivity. Results. On the primary target, Stage1-EXCLUDE concordance was PABAK = AC1 = 1.000 versus E.P. (48/48) and versus D.C. (50/50), and PABAK = 0.940 / AC1 = 0.959 versus Berti (48/50 EXCLUDE, 2 UNCERTAIN). That stratum is 670/904 records (74.1%). Pairwise human agreement was highest between the two external readers (Costazza vs Berti: PABAK = 0.854, AC1 = 0.881) and lower when E.P. was one of the pair (0.703 / 0.774 vs Costazza; 0.621 / 0.702 vs Berti). AI versus each human remained poor (PABAK 0.225 / 0.205 / 0.222). Majority labels on 185 records were 146 EXCLUDE / 27 UNCERTAIN / 4 INCLUDE / 8 NO_MAJORITY. The machine labelled INCLUDE 0 of the 4 majority-INCLUDE records (74, 75, 76, 80). Overall sensitivity was 0.800 versus E.P. (8/10; false negatives 623, 712), 0.500 versus D.C. (1/2; false negative 76), and 0.833 versus Berti (5/6; false negative 76). Versus majority the false-negative count is 1 (record 76); records 623 and 712 are majority EXCLUDE. Stage1-INCLUDE sensitivity of 1.000 is tautological. WSS = 74.1% describes the exclusion stratum only. Conclusions. What this study validates, now with three independent physicians, is automatic exclusion at Stage 1. Fine classification of INCLUDE records is not validated: humans do not agree on which residual records are INCLUDE (triple INCLUDE intersection = record 74 only), Stage 2 missed most of the few human INCLUDE labels it received, and the machine recovered none of the four majority-INCLUDE records. Domain-familiarity as an explanation of INCLUDE divergence remains a hypothesis, not a measured variable.

Background: Ambient artificial-intelligence (AI) scribes are being deployed beyond medicine into allied-health practice, but comparative evidence mainly concerns physician documentation. Objective: To compare quality and transcript-grounded clinical-risk error burden of AI-generated and allied health professional notes from identical simulated consultations. Methods: Paired simulation across dietetics, physiotherapy, occupational therapy and speech and language therapy (five consultations each; 20 pairs; 40 notes). Heidi generated an unedited first-draft note and a professional documented each encounter. Professionals rated PDQI-9 quality. Professional and automated review identified unsupported statements and omissions; two language models graded clinical risk. Co-primary outcomes were PDQI-9 score and Critical+High error density. Results: Mean PDQI-9 scores were 38.45 and 35.05 for AI and professional notes (paired difference +3.40, 95% CI +0.01 to +6.79; Wilcoxon p=0.042; dz=0.47). The Bayesian estimate was +3.47 (94% equal-tailed credible interval +0.66 to +6.30; P[AI>professional]=0.989). Automated-reviewer all-error density was 3.35 vs 8.55 per note (professional-minus-AI difference 5.20, 95% CI 3.85 to 6.55); omission density was 1.45 vs 5.20 (difference 3.75, 2.80 to 4.65). Critical+High density was 0.35 vs 0.75 (difference 0.40, -0.10 to 1.00). The pooled quality result was driven principally by dietetics. Conclusions: AI-generated notes achieved modestly higher pooled quality and fewer transcript-grounded errors, chiefly omissions; serious-error evidence was inconclusive. Small discipline samples and incomplete ground truth preclude safety or discipline-wide superiority claims.

Background Population-level administrative data cohorts hold huge potential for generating evidence to improve child health. Understanding the quality of these data is critical to enabling us to fully realise this potential. We aimed to evaluate linkage between children's health, education and social care administrative records (ECHILD) to identify possible sources of bias. Methods We created four cohorts to understand the drivers of inclusion/exclusion within ECHILD. For each cohort, we calculated the linkage rate by counting the total number of eligible individuals who appeared in the ECHILD linkage spine and any of the corresponding hospital/school component datasets. We then assessed whether linkage rates varied according to sociodemographic characteristics (year of birth, sex, ethnicity, region). Results ECHILD currently captures 34.2 million patients and 25.3 million pupils born between September 1984 and March 2023, aged 0-38 years. Over 90% of children with hospital birth records linked to a school record (rising to 93% for those born from 2014 onwards); 88% of children attending school in 2001/02 linked to a patient record (rising to 96% for those born in 2019/20), resulting in a total of 21 million linked individuals. Substantial variation existed between sociodemographic groups, with those living in London, residing in more deprived areas, or with recorded ethnicity other than White being less likely to be included. Conclusions Population-level datasets such as ECHILD often under-represent specific groups. Irrespective of the mechanisms by which individuals are excluded (e.g. opt outs, linkage errors or not interacting with services), those in minoritised ethnic groups and those living in more deprived areas are most affected. Linkage evaluations are critical for understanding who is and who is not included in analyses of these data so that researchers can account for potential sources of bias within analyses.

Clinicians routinely make decisions for patients whose comorbidities, prior therapies, laboratory abnormalities, or demographic characteristics are not reflected in the supporting evidence, even though these factors can influence treatment selection. Given this gap in the evidence base, we sought to evaluate how AI models respond to clinical questions in which patient characteristics or the treatment comparison and outcome of interest may affect the applicability of available evidence. We developed and publicly released Precision Evidence Bench, a benchmark of 209 synthetic clinical questions with specified population, intervention, comparator, and outcome (PICO) elements. Each answer is assessed using a five item rubric that evaluates whether a single cited source matches the PICO and whether the answer faithfully reports its sources. The item verdicts determine one of five ratings, ranging from Precision match to Unsupported claim, without requiring a reference answer. We evaluated four frontier large language models (LLMs) using web search across all 209 questions, then repeated the evaluation with additional access to a corpus of precision evidence-based findings (pEBFs) designed to address these evidence gaps, using two corpus sizes. For every model, Precision match rates increased with the amount of precision evidence available, from 13 to 18% with web search alone to 17 to 28% with 100 million pEBFs and 41 to 56% with 500 million pEBFs. These findings suggest that access to matching evidence is an important determinant of LLM performance on precision medicine questions.

As digital self-triage becomes more widely used, quality assurance requires evaluation of both emergency recognition and overall agreement with clinical judgement, including how these outcomes differ across generations of large language models. The Benchmark Self-Triage (BeST) project intends to provide a framework to compare systems for digital self-triage against a medical 'gold standard' in Germany. In a first step developing the method and demonstrating its potential we developed a representative set of patient vignettes and compared two regulated medical-device triage systems, SmED and Infermedica, with five structured OpenAI configurations spanning multiple model generations: gpt-4.1, gpt-5.4, gpt-5.6-luna, gpt-5.6-sol, and gpt-6-astra. An LLM-based patient actor covered 480 physician-curated vignettes derived from German routine triage data. A pool of seven physicians provided preliminary reference classifications and consensus for 480 vignettes. Recommendations were mapped to three urgency categories 'emergency', 'acute', and 'not urgent' and evaluated using winsorised weights derived from routine triage frequencies. Weighted emergency sensitivity ranged from 51.3% (95% CI 35.5-66.3%) for gpt-4.1 to 99.9% (99.7-100.0%) for gpt-6-astra. Weighted exact three-category agreement (accuracy) ranged from 54.4% to 68.8%; gpt-4.1 achieved the highest agreement despite the lowest emergency sensitivity. No system performed best across all urgency categories. The later-generation OpenAI configurations showed a more precautionary triage profile resembling SmED, with higher sensitivity for emergency cases; gpt-6-astra showed the strongest shift. This cross-sectional comparison across model generations suggests a shift towards more precautionary triage behaviour, without establishing a longitudinal improvement in accuracy. These findings highlight the limitations of overall agreement for evaluating self-triage and support category-specific assessment. Broader expert consensus and evaluation, as well as further steps to reduce dependence on SmED-derived source data, are needed and planned before drawing conclusions about clinical safety.

Background: Vitamin B12 is essential for one-carbon metabolism, deoxyribonucleic acid (DNA) synthesis, and myelination, and inadequate maternal status during pregnancy may influence fetal growth and child neurodevelopment. Objective: To evaluate the association between maternal vitamin B12 status and perinatal and child neurodevelopmental outcomes. Methods: We conducted a systematic review and meta-analysis of observational studies and randomized clinical trials. The main outcomes were preterm birth, low birth weight, small for gestational age (SGA), and cognitive/global development. For binary outcomes, the natural logarithm of the relative risk (logRR) was the primary analytical effect measure; for neurodevelopment, Hedges g was used. Random-effects models were fitted, between-study variance was estimated by restricted maximum likelihood (REML), and inference used Knapp-Hartung (KH) adjustment. Results: For preterm birth, nine estimates yielded a pooled logRR of 0.190 (95% confidence interval [95% CI] KH: -0.046 to 0.427; p = 0.101; I2 = 19.2%). For low birth weight, nine estimates yielded a pooled logRR of 0.136 (95% CI KH: -0.013 to 0.286; p = 0.069; I2 = 4.8%). For SGA, six estimates yielded a pooled logRR of 0.047 (95% CI KH: -0.083 to 0.177; p = 0.397; I2 = 0%). Cognitive/global development, assessed in five independent studies, showed a pooled Hedges g of 0.266 (95% CI KH: 0.034 to 0.499; p = 0.034; I2 = 55.7%). Conclusions: logRR estimates for preterm birth and low birth weight were directionally compatible with higher risk among mothers with lower vitamin B12 availability, but their 95% CIs included the null. No consistent association was observed for SGA. Higher maternal vitamin B12 availability or supplementation was associated with a small favorable effect on cognitive/global development, with moderate heterogeneity.